Method and device for simulating a wide field of view

ABSTRACT

It is disclosed a method for displaying input image data on a display device. The display device comprises a main display and a border display which at least partly surrounds the main display. The method comprises extrapolating the input image data to obtain extrapolated image data, displaying at least part of the input image data on the main display, and displaying at least part of the extrapolated image data on the border display.

TECHNICAL FIELD

The present disclosure generally pertains to a method and an electronicdevice for simulating a wide field of view. The method and electronicdevice may be used in a display device, e.g. in a head mounted display,to display video or image data.

TECHNICAL BACKGROUND

One possible application of Head Mounted Displays (HMDs) is the“Personal Display” (PD). A Personal Display is a Head Mounted Displaythat is tailored for watching movies or similar content. However, thefield of view (FOV) of video or image data can be limited to ˜30degrees, since the video content is usually shot with this field of viewin mind, i.e. for normal video content it is assumed that the viewingdistance from the screen is proportional to the screen diagonal so thatthe field of view is ˜30 degrees. While this field of view fitsperfectly the requirements for normal television screens, it may stillbe uncomfortable to view movies at a field of view of ˜30 degreeslooking through a Personal Display and to have the surroundingenvironment fully black.

SUMMARY

According to a first aspect the disclosure provides a method fordisplaying input image data on a display device, the display devicecomprising a main display and a border display which at least partlysurrounds the main display, the method comprising extrapolating theinput image data to obtain extrapolated image data, displaying at leastpart of the input image data on the main display, and displaying atleast part of the extrapolated image data on the border display.

According to a second aspect, the disclosure provides a display device,the display device comprising a main display, a border display which atleast partly surrounds the main display, and a processor, configured toextrapolate input image data to obtain extrapolated image data, displayat least part of the input image data on the main display, and displayat least part of the extrapolated image data on the border display.

Further aspects are set forth in the dependent claims, the followingdescription and the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are explained by way of example with respect to theaccompanying drawings, in which:

FIG. 1 schematically shows an embodiment of a display device;

FIG. 2 a schematically shows an embodiment of a processor;

FIG. 2 b schematically shows an alternative embodiment of a processor;

FIG. 3 schematically shows in more detail an embodiment of a maindisplay and a border display.

FIG. 4 a shows an embodiment of a scaling function sx;

FIG. 4 b shows an embodiment of an inverse scaling function sx⁻¹;

FIG. 5 schematically shows the result of a mapping;

FIG. 6 shows an exemplifying result of the multiscale foveated videoextrapolation technique;

FIG. 7 shows an embodiment of a method for displaying input image dataon a display device; and

FIG. 8 shows an embodiment of a head mounted display comprising thedisplay device.

DETAILED DESCRIPTION OF EMBODIMENTS

Before a detailed description of the embodiments under reference of FIG.1, general explanations are made.

A current Personal Display (PD) usually delivers a narrow field of view(FOV) of about 30 degrees. This fits the requirements, but since thedisplay is head mounted (HMD), all the surroundings are usually black.The human vision system (HVS) does not have good spatial resolution atthe peripheral; nevertheless, it is very sensible to motion, light andcolour changes. That is why it can be uncomfortable to watch contentwith static black surroundings. It might thus be more comfortable tohave content in the surrounding areas, too. But as normal video or imagedata usually is made for a field of view of ˜30 degrees, content for thesurrounding areas must be created so that it covers these areas.

Here techniques are disclosed which help to fill these “surroundings”with some simulated content, correlated with the content of the centralview.

In particular, a method it is disclosed for displaying input image dataon a display device, the display device comprising a main display and aborder display which at least partly surrounds the main display. Themethod comprises extrapolating the input image data to obtainextrapolated image data, displaying at least part of the input imagedata on the main display, and displaying at least part of theextrapolated image data on the border display.

Input image data may comprise video images or still images. The inputimage data may correspond to non-compressed or compressed video imagedata such as MPEG video data, or non-compressed or compressed videoimage data such as JPG, GIF, or the like.

In the embodiments, the main display is arranged for foveal viewing andthus preferably provides a good screen resolution.

In the embodiments, the border display is arranged for parafovealviewing and thus is located in the area surrounding the main display.The border display may have a screen resolution that is the same ordifferent from that of the main display. As the border display isarranged for parafoveal viewing, the border display may in particularhave a smaller screen resolution than the main display.

The main display and the border display may for example be LiquidCrystal Displays (LCD) or Plasma Display Panels (PDP) of given screenresolutions, or the like. The border display may even be comprised ofonly a small number of LEDs that provide a very low screen resolution.

The main display and the border display may be flat or curved. The maindisplay and the border display may have rectangular shapes or curvedshapes, such as an oval shape. In the embodiments disclosed, the maindisplay has a rectangular shape and the border display has a frame likeshape and completely surrounds the main display.

In the disclosed embodiments, the input image data is bitmap data thatrepresents an image with a resolution which fits to that of the maindisplay. In other embodiments, the input image data may have aresolution different to that of the main display. In such cases, theinput image data may be scaled to fit the screen resolution of the maindisplay with techniques known in the art.

Extrapolating the input image data may be based on part of the inputimage data or the full input image data. In the disclosed embodiments,extrapolating the input image data is not only based on border pixels ofthe input image, but is also based on input image data which representsinner parts of the input image. This allows that structures, texturesand even objects can be extrapolated to produce content for displayingon the border display.

According to an embodiment, extrapolating the input image data maycomprise scaling the input image data.

The scaling of the input image data may be based on a non-linear scalingalgorithm. Scaling the input image data may result in that at leastparts of the input image data are extrapolated for displaying on theborder display and that at least other parts of the input image dataremain for displaying on the main display.

For example, extrapolating the input image data may be based on themapping functions

x′=sx(|x−X1/2|)*(x−X1/2)+X2/2

y′=sy(|y−Y1/2|)*(y−Y1/2)+Y2/2

where x and y are pixel coordinates describing the location of a pixelon the main display, x′ and y′ are pixel coordinates describing thelocation of a pixel on the border display, X1 and Y1 denote thedimensions of the main display in pixels, X2 and Y2 denote thedimensions of the border display in pixels, and sx and sy are predefinedscaling functions.

According to other embodiments, extrapolating the input image data isbased on a multiscale foveated video extrapolation technique.

Other techniques such as using trained filters or a linear scaling mightalso be used for extrapolating the input image data.

The method may further comprise detecting and removing black borders inthe input image data.

Further, a display device is disclosed, the display device comprising amain display, a border display which at least partly surrounds the maindisplay, and a processor. The processor is configured to extrapolateinput image data to obtain extrapolated image data, display at leastpart of the input image data on the main display, and display at leastpart of the extrapolated image data on the border display.

In the display device the main display may be arranged to provide anormal field of view of ˜30 degrees and the border display may bearranged to enlarge the field of view above 30 degree, for example up to100 degrees, or even up to ˜200 degrees or more.

Further, a head mounted display is disclosed, the head mounted displaycomprising the above described display device. When applied in a headmounted display, the main display of the electronic device typicallycomprises a first main display and a second main display for displayinginput image data to a left eye and, respectively, to a right eye, andthe border display of the electronic device comprises a first borderdisplay and a second border display for displaying input image data to aleft eye and, respectively, to a right eye.

Display Device

FIG. 1 schematically shows an embodiment of a display device 1. Thedisplay device 1 comprises a main display 3, and a border display 5which at least partly surrounds the main display 3. A processor 7 isconfigured to receive input image data 9. The input image data 9 relatesto a media file or media stream, such as video data. In the case ofvideo data, each video frame may be independently processed as inputimage data 9. Alternatively, multiple video frames may be processedtogether as input image data.

The processor is further configured to extrapolate the input image data9 to obtain extrapolated image data 15. The processor displays at leastpart of the input image data 9 on the main display 3. Further, theprocessor 7 displays at least part of the extrapolated image data 15 onthe border display 5.

FIG. 2 a schematically shows an embodiment of processor 7. The processorcomprises an image analysis unit 21 which is arranged to receive inputimage data 9. The image analysis unit 21 is configured to determinecharacteristics of the input image data 9, such as image resolution,etc. For example, the input image data 9 may be MPEG4-coded video datawith an image resolution of 1280×720 pixels.

The image analysis unit 21 may also be configured to perform otherprocesses to analyze the input image data 9. For example, the imageanalysis unit 21 may be configured to implement a black border detectionprocess that detects black border information 19 which identifies pixelsin the input image data 11 which relate to black borders.

The image analysis unit 21 passes the input image data 9, the detectedimage resolution 17, and any black border information 19 toextrapolation unit 23. Extrapolation unit 23, if necessary, removesblack border pixels based on the black border information 19 receivedfrom the image analysis unit 21, and extrapolates the input image data 9to obtain extrapolated image data 15. This extrapolation is described inmore detail below with reference to FIGS. 3, 4 a, 4 b, 5, and 6. Theextrapolation unit 23 passes the extrapolated image data 15 to a borderdisplay driver 25. The border display driver 25 converts theextrapolated image data 15 into respective electronic signals 15′ whichare passed to border display 5. Border display 5 is configured toreceive the electronic signals 15′ from border display driver 25 and todisplay the corresponding extrapolated image data 15.

The image analysis unit 21 also passes the input image data 9 and thedetected image resolution 17, and any black border information 19 to amain display processing unit 27. This main display processing unit 27processes the input image data 9 to obtain main image data 13. Forexample, the main display processing unit 27 may be configured to removeany image data from the input image data 9 that is identified by theblack border information 19 as relating to black borders. The maindisplay processing unit 27 passes the main image data 13 to a maindisplay driver 29. The main display driver 29 converts the main imagedata 13 into respective electronic signals 13′ which are passed to maindisplay 3. Main display 3 is configured to receive the electronicsignals 13′ from main display driver 29 and to display the correspondingmain image data 13. In the embodiments described below it can be assumedthat the main display data 13 and the input image data 9 are at least tosome extent the same.

In the embodiment of FIG. 2 a, two separate units, namely extrapolationunit 23 and main display processing unit 27, are shown whichindependently generate the extrapolated image data 15 and the main imagedata 13. These units must not necessarily be separate physical entities.FIG. 2 b shows an alternative embodiment, in which a single unit, namelyextrapolation unit 23, is configured to process the input image data 9and to generate the extrapolated image data 15 and the main image data13 from this input image data 9.

Likewise, despite that in FIGS. 2 a and 2 b the image analysis unit 21,the extrapolation unit 23, the main display processing unit 27, theborder display driver 25 and the main display driver 29 are shown asseparate units, the functionality of these units must not necessarily beperformed by separate physical entities. In general processor 7 of FIG.7 can be configured in many alternative ways to provide thefunctionality disclosed in this application.

FIG. 3 schematically shows in more detail an embodiment of main display3 and border display 5.

Main display 3 is a rectangular LCD display with a screen resolution ofX1,Y1 pixels. (x,y)=(0,0) describes the pixel which is located in theupper-left corner of the display. (x,y)=(X1,Y1) describes the pixelwhich is located in the lower-right corner of the display.

Border display 5 is a rectangular LCD display with a screen resolutionof X2,Y2 pixels. (x′,y′)=(0,0) describes the pixel which is located inthe upper-left corner of the display. (x′,y′)=(X2,Y2) describes thepixel which is located in the lower-right corner of the display. Borderdisplay 5 has a frame-like shape. That is, it surrounds main display 3and does not comprise any pixels in the center region which is alreadycovered by pixels of the main display 3.

In this embodiment, the screen resolution of the border display 5 issmaller than that of the main display 3. For example, the main display 3might have a screen resolution of X1, Y1=1280×720 pixels and the borderdisplay 5 might have a screen resolution of X1, Y1=640×480 pixels, oreven less. In other embodiments, the screen resolution of the borderdisplay 5 may be the same as that of main display 3.

The main display 3 provides a field of view of ˜30 degrees which is thefield of view normal video content is typically produced for. The borderdisplay 5 extends this field of view of ˜30 degrees to a larger field ofview. FIG. 3 gives only an exemplifying and schematic representation ofthe two displays. The border display 5 might for example extend thefield of view to values such as ˜200 degrees. To achieve such a largefield of view, the border display 5 could be, other than depicted in theembodiment of FIG. 3, much larger than the main display 3.

The input image data 9 of FIG. 1 is typically designed for a field ofview of ˜30 degrees, and thus for displaying on the main display 3. Forexample, the resolution of the input image data 9 might have aresolution of 1280×720 pixels and thus fit to the resolution of the maindisplay 3. Such input image data 9 would not yet contain image data fordisplaying at a larger field of view, i.e. on the border display 5.Image data for displaying on the border display 5 is generated byextrapolation of the input image data 9 in extrapolation unit 23.

Extrapolation by Non-Linear Scaling

FIGS. 4 a and 4 b schematically describe in more detail an embodiment offunctionality which is performed by extrapolation unit 23 of FIG. 2 a orFIG. 2 b.

According to the embodiment of FIGS. 4 a, and 4 b the extrapolation unit23 performs a non-linear scaling process on the input image data 9 toproduce extrapolated image data 15 for displaying on border display 5.The non-linear scaling process is schematically represented by a scalingfunction s.

The scaling function s is configured to map the input image (which isdesigned for display on the main display 3) so that it covers the fieldof view of the border display 5, i.e. to map the pixels (x,y) of themain display 3, i.e. the dimension X1, Y1, to the pixels (x′, y′) andthe dimensions X2, Y2 of the border display 5.

In the embodiment described below, it is assumed that the pixelresolution of the border display 5 is the same as the pixel resolutionof main display 3.

According to the embodiment described here, the scaling function isdecomposed into two parts sx and sy, where sx is applied to the xcoordinates and sy is applied to y-coordinates, according to:

x′=sx(|x−X1/2|)*(x−X1/2)+X2/2   eq. (1)

y′=sy(|y−Y1/2|)*(y−Y1/2)+Y2/2   eq. (2)

The scaling functions sx(|x−X1/2|) and sy(|y−Y1/2|) describe scalingfactors. The principles of scaling are described here with regard to thex-dimension part sx of the scaling function only. For the y-dimensionpart of the scaling function the same principles do apply.

FIG. 4 a shows an embodiment of a scaling function sx. The scalingfunction sx receives a shifted version of the input image data 9 inwhich the input image is shifted by −X1/2. This means that the scalingfunction sx is centered at the center of the input image, respectivelythe center of the main display 3 which is at x=X1/2. Further, thescaling function sx according to this embodiment is symmetric withrespect to the center, which means that s(x)=s(−x). FIG. 4 a thusdepicts s(|x−X1/2|), where x is the x-coordinate of the input image.

As can be seen in FIG. 4 a, for x=X1/2, the scaling function issx(|x−X1/2=s(0)=1. This means that according to eq. (1) the centerx=X1/2 of the main display 3 is mapped to x′=X2/2, i.e. to the (virtual)center of the border display 5.

Further, as can be seen in FIG. 4 a, for x=X1 (i.e. for pixels at theright boundary of the main display 3) the scaling function issx(|x−X1/2|)=sx(X1/2)=X2/X1. According to eq. (1), such pixels aremapped to x′=sx(X1−X1/2)*(X1−X1/2)+X2/2=X2/X1*X1/2+X2/2=X2. That is,pixels at the right boundary of the main display 3 are mapped to pixelsat the right boundary of the border display 3.

Further, as can be seen in FIG. 4 a, for x=0 (i.e. for pixels at theleft boundary of the main display 3) the scaling function issx(|x−X1/2|)=sx(X1/2)=X2/X1. According to eq. (1), such pixels aremapped to x′=sx(X1/2)*(0−X1/2)+X2/2=X2/X1*(−X1/2)+X2/2=0. That is,pixels at the left boundary of the main display 3 are mapped to pixelsat the left boundary of the border display 3.

Further, as can be seen in FIG. 4 a, the scaling function sx has a valueof substantially 1 in the interval [0,x0] which corresponds to a centerregion of the input image. This means that x′=x in a center region ofthe input image. In other words, in the interval [0,x0] of the scalingfunction sx the scaling factor sx is substantially 1 so that the centerof the image remains substantially unscaled. The scaling function sxthen increases in a strictly monotonic way until the scaling factorreaches its maximum X2/X1 for |x−X1/2|=X1/2, i.e. at the left and rightboundaries x=0 and x=X1 of the main display 3 (respectively the inputimage 9).

FIG. 4 b shows an embodiment of an inverse scaling function sx⁻¹. Theinverse scaling function sx⁻¹ is configured to do the inverse of thescaling function sx. It is configured to map pixels of the borderdisplay 5 to pixels of the main display 3, i.e. to map the pixels(x′,y′) of the border display 5, i.e. the dimension X2, Y2, to thepixels (x, y) and the dimensions X1, Y1 of the main display 3:

x=sx ⁻¹(|x′−X2/2|)*(x′−X2/2)+X1/2 eq. (3)

y=sy ⁻¹(|y′−Y2/2|)*(y′−Y2/2)+Y1/2 eq. (4)

Let (x′,y′) describe a pixel of the border display 5, then equations (3)and (4) can be used to determine the pixel (x,y) of the main display 3which in the scaling process maps to this pixel (x′,y′). Thus, accordingto equations (3) and (4), the color c(x′,y′) of a pixel (x′,y) of borderdisplay 3 can be obtained from the color c(x,y) of the correspondingpixel (x,y) of the main display 3 as c(x′,y′)=c(x,y). As the input imagedata 9 and the main display 3 are both designed for a field of view of−30 degrees, the color c(x,y) of pixel (x,y) of the main display 3 candirectly be identified in the input image data 9. It is thus describedin this embodiment a process of extrapolating the input image data 9 toobtain extrapolated image data for displaying on the border display 5using a non-linear scaling function.

FIG. 5 schematically shows the result of the mapping of eq. (1). On theabscissa of the coordinate system it is plotted x−X1/2, i.e. the originof the coordinate system of FIG. 5 is positioned in the center of themain display 3. On the ordinate of the coordinate system it is plottedsx(x−X1/2)*(x−X1/2), which is the result of the mapping of equation (1),shifted by −X2/2, i.e. with the origin of the coordinate systempositioned in the center of border display 5.

As the values plotted on the abscissa equal to sx⁻¹(x′−X2/2) and thevalues plotted on the ordinate equal to x′−X2/2, the graph of FIG. 5,can also be seen as displaying the result of the inverse mapping ofequation (3), point mirrored at the origin of the coordinate system. Theinverse scaling function sx⁻¹ can thus be obtained from the scalingfunction sx by mirroring and numerical evaluations.

It can be seen in FIG. 5 that the mapping functions of eq. (1) and (3)leave the center of the image substantially unsealed.

The same principles can be applied in embodiments, in which the pixelresolution of the border display 5 is smaller than the pixel resolutionof main display 3. In such embodiments, the equations shown above can beadapted to the pixel resolution of the border display. Alternatively,for the purpose of the mapping, the border display 5 can be assumed to(virtually) have the same pixel resolution as the main display 3, and,after having performed the non-linear scaling as described above, alinear downscaling process can be applied to adapt the result of thenon-linear scaling to the specific pixel resolution of the borderdisplay 5.

Multiscale Foveated Video Extrapolation

According to other embodiments, extrapolating the input image data isbased on a multiscale foveated video extrapolation technique such asdescribed by Amit Aides et al in “Multiscale Ultrawide Foveated VideoExtrapolation”, Proc. IEEE Int. Conference on Computational Photography(ICCP), 2011.

The multiscale foveated video extrapolation technique relates to videoinpainting, which is also known as video completion. As described in theabove reference, in video inpainting, the missing parts are inside thefield of view, while the multiscale foveated video extrapolationtechnique extrapolation technique aims at extending the video beyond thefield of view. The multiscale foveated video extrapolation techniqueimitates the behavior of the human fovea by diminishing the resolutiontowards the boundaries of the extrapolated region.

As described in more detail by Amit Aides et al, the multiscale foveatedvideo extrapolation technique applies a so called multistep “Outside-In”approach. The extrapolation starts with the coarsest scale, processingthe whole extrapolation domain. Then the other resolutions scales areused consecutively from coarse to fine, where a finer scale applies to asmaller (inner) domain. The “Outside-In” method uses the informationextrapolated in coarse scales to initialize extrapolation at finerscales. The inner domains of the output video are processed severaltimes, in different resolution scales, gradually refining the result.

FIG. 6 shows an exemplifying result of the multiscale foveated videoextrapolation technique. An inner domain image 61 is extrapolated toproduce an outer domain image 63. The outer domain image 63 showsstructures which resemble structures found in the inner domain image 61.

In alternative embodiments, other video extrapolation techniques such asthose based on an “Inside-Out” approach are used. As described in moredetail by Amit Aides et al, the “Inside-Out” approach, starts at thefinest resolution level, i.e. the innermost domain, and proceedsoutwards from fine to coarse.

Other inpainting approaches may also be adapted for use in extrapolatingthe input image data to obtain extrapolated image data.

Method for Displaying Input Image Data

In FIG. 7 it is shown an embodiment of a method for displaying inputimage data on a display device, the display device comprising a maindisplay and a border display which at least partly surrounds the maindisplay. At S701, input image data is extrapolated to obtainextrapolated image data. At S703, at least part of the input image datais displayed on the main display. At S705, at least part of theextrapolated image data is displayed on the border display. Theextrapolating at S703 can for example be performed as described above,i.e. by non-linear scaling, by multiscale foveated video extrapolation,or by other extrapolation techniques.

Head Up Display

In FIG. 8 it is shown an embodiment of a head mounted display 80comprising the display device as described above. A main display 3 l, 3r of the electronic device comprises a first main display 3 l and asecond main display 3 r for displaying input image data to a left eyeand, respectively, to a right eve. A border display 5 l, 5 r of theelectronic device comprises a first border display 31 and a secondborder display 3 r for displaying input image data 9 to a left eye and,respectively, to a right eye. A processor 7 is arranged to process theinput image data 9 to produce extrapolated image data 15 for displayingon the border display 5 l, 5 r and main image data 13 for displaying onthe main display 3 l, 3 r.

All units and entities described in this specification and claimed inthe appended claims can, if not stated otherwise, be implemented asintegrated circuit logic, for example on a chip, and functionalityprovided by such units and entities can, if not stated otherwise, beimplemented by software.

It is disclosed in this application:

[1]. A method for displaying input image data on a display device, thedisplay device comprising a main display and a border display which atleast partly surrounds the main display, the method comprising

-   -   extrapolating the input image data to obtain extrapolated image        data,    -   displaying at least part of the input image data on the main        display, and displaying at least part of the extrapolated image        data on the border display.

[2]. The method of [1], in which extrapolating the input image datacomprises scaling the input image data.

[3]. The method of [1] or [2], in which extrapolating the input imagedata is based on a non-linear scaling algorithm.

[4]. The method of anyone of [1], [2] or [3], in which extrapolating theinput image data is based on the mapping functions

x′=sx(|x−X1/2|)*(x−X1/2)+X2/2

y′=sy(|y−Y1/2|)*(y−Y1/2)+Y2/2

-   -   where    -   x and y are pixel coordinates describing the location of a pixel        on the main display,    -   x′ and y′ are pixel coordinates describing the location of a        pixel on the border display,    -   X1 and Y1 denote the dimensions of the main display in pixels,    -   X2 and Y2 denote the dimensions of the border display in pixels,        and sx and sy are predefined scaling functions.

[5]. The method of [1], in which extrapolating the input image data isbased on a multiscale foveated video extrapolation technique.

[6]. The method of anyone of [1] to [5], further comprising detectingand removing black borders in the input image data.

[7]. A display device, the display device comprising

-   -   a main display,    -   a border display which at least partly surrounds the main        display, and    -   a processor, configured to        -   extrapolate input image data to obtain extrapolated image            data,        -   display at least part of the input image data on the main            display, and        -   display at least part of the extrapolated image data on the            border display.

[8]. The display device of [7] in which the main display is arranged toprovide a normal field of view of about 30 degrees and in which theborder display enlarges the field of view above 30 degree, or preferablyup to 100 degrees, or more preferably up to about 200 degrees or more.

[9]. A head mounted display comprising the display device of [7] or [8],in which

-   -   the main display of the electronic device comprising a first        main display and a second main display for displaying input        image data to a left eye and, respectively, to a right eye, and    -   the border display of the electronic device comprising a first        border display and a second border display for displaying input        image data to a left eye and, respectively, to a right eye.

1. A method for displaying input image data on a display device, thedisplay device comprising a main display and a border display which atleast partly surrounds the main display, the method comprisingextrapolating the input image data to obtain extrapolated image data,displaying at least part of the input image data on the main display,and displaying at least part of the extrapolated image data on theborder display.
 2. The method of claim 1, in which extrapolating theinput image data comprises scaling the input image data.
 3. The methodof claim 2, in which extrapolating the input image data is based on anon-linear scaling algorithm.
 4. The method of claim 3, in whichextrapolating the input image data is based on the mapping functionsx′=sx(|x−X1/2|)*(x−X1/2)+X2/2y′=sy(|y−Y1/2|)*(y−Y1/2)+Y2/2 where x and y are pixel coordinatesdescribing the location of a pixel on the main display, x′ and y′ arepixel coordinates describing the location of a pixel on the borderdisplay, X1 and Y1 denote the dimensions of the main display in pixels,X2 and Y2 denote the dimensions of the border display in pixels, and sxand sy are predefined scaling functions.
 5. The method of claim 1, inwhich extrapolating the input image data is based on a multiscalefoveated video extrapolation technique.
 6. The method of claim 1,further comprising detecting and removing black borders in the inputimage data.
 7. A display device, the display device comprising a maindisplay, a border display which at least partly surrounds the maindisplay, and a processor, configured to extrapolate input image data toobtain extrapolated image data, display at least part of the input imagedata on the main display, and display at least part of the extrapolatedimage data on the border display.
 8. The display device of claim 7 inwhich the main display is arranged to provide a normal field of view ofabout 30 degrees and in which the border display enlarges the field ofview above 30 degree, or preferably up to 100 degrees, or morepreferably up to about 200 degrees or more.
 9. A head mounted displaycomprising the display device of claim 7, in which the main display ofthe electronic device comprising a first main display and a second maindisplay for displaying input image data to a left eye and, respectively,to a right eye, and the border display of the electronic devicecomprising a first border display and a second border display fordisplaying input image data to a left eye and, respectively, to a righteye.